Papers with cost-effective tree search approach
TreeRL: LLM Reinforcement Learning with On-Policy Tree Search (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for On-Policy LLM RL typically train a separate process reward model, which suffers from distribution mismatch and reward hacking. |
| Approach: | They propose a reinforcement learning framework that directly incorporates on-policy tree search for RL training. |
| Outcome: | Experiments on math and code reasoning benchmarks show that tree search achieves superior performance compared to traditional ChainRL. |